The present invention relates generally to real-time event processing, and more particularly to methods, apparatus and systems for processing real-time events in applications such as telecommunications and computer networks.
Many event-processing systems have real-time performance requirements that cannot be met by conventional general-purpose database management systems. In telecommunications applications, for example, a variety of adjunct switching services such as debit-based billing, number mapping, call forwarding, and local-number portability involve event processing during the critical call-connection phase of a telephone call. To meet the real-time requirements of the network, the service time for such events generally must not exceed a few milliseconds. However, with conventional database technology, the costs of invoking a structured query language (SQL) operation over a client-server interface, or the costs associated with a single access to secondary storage, can account for hundreds of milliseconds. As a consequence, performance goals on the order of a few milliseconds may already be unattainable even before the costs of the event processing logic are taken into account.
These limitations have led to the use of custom database systems for many high-performance real-time event processing applications. Custom systems are typically tightly coupled to their particular applications, and are tuned to the specific requirements of those applications. These solutions generally work well in practice, and can provide close to optimal performance. However, they also have a number of serious disadvantages. For example, the cost of developing and maintaining custom systems can be high, and generally cannot be amortized across a number of different applications. Moreover, custom systems are frequently inflexible. It can become difficult or even impossible to adapt a custom system to unforeseen or evolving requirements.
A need therefore exists for an improved real-time event processing system which provides the performance benefits of custom database systems, without sacrificing the flexibility and maintainability typically associated with conventional general-purpose database systems.
The invention provides a general-purpose real-time event processing system (EPS) which avoids the problems associated with custom systems. The EPS in an illustrative embodiment meets real-time performance goals through the use of one or more real-time analysis engines (RAEs) operating in conjunction with a main-memory storage manager as its underlying database system. The main-memory storage manager offers transactional access to persistent data, but at the speed of a main-memory system. The EPS may implement a parallel arrangement of RAEs which is readily scalable as workload and resources increase. The RAEs may also store subscription information in accordance with a subscription model that serves as a basis for flexible, low-overhead, real-time event processing. Application-specific services for execution in the RAEs of the EPS may be generated in a service authoring environment (SAE). The SAE provides a set of tools and graphical user interfaces (Guls) for authoring, validating, compiling and installing new services, or adapting existing services to new requirements. The EPS may also store recovery information, e.g., in a memory portion of the main-memory database system, in order to facilitate roll-back to a recovery point after an RAE failure.
In accordance with one aspect of the invention, an EPS may include a set of RAEs operating in parallel, e.g., a set of clusters each including one or more RAEs, and one or more mappers for mapping a given input event to a particular one of the clusters. A main-memory database system is coupled to the RAEs, and the RAEs process events associated with input streams from one or more data sources, and deliver output streams to one or more data sinks. The data sources and sinks may be, e.g., network elements, clients, databases, etc. The events are processed in accordance with services implemented in the RAEs, and utilize data stored in a memory portion of the main-memory database system accessible to the RAEs. The data may include, e.g., a subscription table storing subscription information indicating the service or services that should be executed for a given event, in accordance with the above-noted subscription model. The services are generated in the SAE associated with the EPS, using a declarative language referred to herein as the service authoring language (SAL). The SAE may be configured to generate the services in the form of object code components, e.g., dynamically linked libraries, which may be dynamically linked into the RAEs without interrupting event processing.
A real-time EPS in accordance with the invention provides a critical path for event processing that is specifically designed for high performance, while also retaining many desirable features of conventional database systems, including high-level, declarative programming interfaces, and the well-known transactional correctness properties of atomicity, consistency, isolation and durability (ACID). These features of the invention enhance the reliability, robustness, usability and maintainability of the real-time EPS and the applications built thereon.
The invention can be used in conjunction with any event processing application, including, for example, telecommunications, electronic commerce, and Internet service provisioning applications. For example, in a telecommunications application, the invention can be configured to provide a basis for features such as enhanced billing systems, fraud detection and prevention, local-number portability, settlements among service providers and real-time traffic analysis.